---
title: TrueFoundry
---

TrueFoundry provides an enterprise-ready [AI Gateway](https://www.truefoundry.com/ai-gateway) to provide governance and observability to agentic frameworks like LangChain. TrueFoundry AI Gateway serves as a unified interface for LLM access, providing:

- **Unified API Access**: Connect to 250+ LLMs (OpenAI, Claude, Gemini, Groq, Mistral) through one API
- **Low Latency**: Sub-3ms internal latency with intelligent routing and load balancing
- **Enterprise Security**: SOC 2, HIPAA, GDPR compliance with RBAC and audit logging
- **Quota and cost management**: Token-based quotas, rate limiting, and comprehensive usage tracking
- **Observability**: Full request/response logging, metrics, and traces with customizable retention


## Prerequisites

Before integrating LangChain with TrueFoundry, ensure you have:

1. **TrueFoundry Account**: A [TrueFoundry account](https://www.truefoundry.com/register) with at least one model provider configured. Follow quick start guide [here](https://docs.truefoundry.com/gateway/quick-start)
2. **Personal Access Token**: Generate a token by following the [TrueFoundry token generation guide](https://docs.truefoundry.com/gateway/authentication)

## Quickstart

You can connect to TrueFoundry's unified LLM gateway through the `ChatOpenAI` interface.

- Set the `base_url` to your TrueFoundry endpoint (explained below)
- Set the `api_key` to your TrueFoundry [PAT (Personal Access Token)](https://docs.truefoundry.com/gateway/authentication#personal-access-token-pat)
- Use the same `model-name` as shown in the unified code snippet

![TrueFoundry metrics](/img/unified-code-tfy.png)

### Installation

<CodeGroup>
```bash pip
pip install langchain-openai
```

```bash uv
uv add langchain-openai
```
</CodeGroup>

### Basic Setup

Connect to TrueFoundry by updating the `ChatOpenAI` model in LangChain:

```python
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    api_key=TRUEFOUNDRY_API_KEY,
    base_url=TRUEFOUNDRY_GATEWAY_BASE_URL,
    model="openai-main/gpt-4o"  # Similarly you can call any model from any model provider
)

llm.invoke("What is the meaning of life, universe and everything?")
```

The request is routed through your TrueFoundry gateway to the specified model provider. TrueFoundry automatically handles rate limiting, load balancing, and observability.

### LangGraph Integration


```python
from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, MessagesState
from langchain_core.messages import HumanMessage

# Define your LangGraph workflow
def call_model(state: MessagesState):
    model = ChatOpenAI(
        api_key=TRUEFOUNDRY_API_KEY,
        base_url=TRUEFOUNDRY_GATEWAY_BASE_URL,
        # Copy the exact model name from gateway
        model="openai-main/gpt-4o"
    )
    response = model.invoke(state["messages"])
    return {"messages": [response]}

# Build workflow
workflow = StateGraph(MessagesState)
workflow.add_node("agent", call_model)
workflow.set_entry_point("agent")
workflow.set_finish_point("agent")

app = workflow.compile()

# Run agent through TrueFoundry
result = app.invoke({"messages": [HumanMessage(content="Hello!")]})
```


## Observability and Governance

![TrueFoundry metrics](/img/gateway-metrics.png)

With the Metrics Dashboard, you can monitor and analyze:

- **Performance Metrics**: Track key latency metrics like Request Latency, Time to First Token (TTFS), and Inter-Token Latency (ITL) with P99, P90, and P50 percentiles
- **Cost and Token Usage**: Gain visibility into your application's costs with detailed breakdowns of input/output tokens and the associated expenses for each model
- **Usage Patterns**: Understand how your application is being used with detailed analytics on user activity, model distribution, and team-based usage
- **Rate Limiting & Load Balancing**: Configure limits, distribute traffic across models, and set up fallbacks

## Support

For questions, issues, or support:

- **Email**: [support@truefoundry.com](mailto:support@truefoundry.com)
- **Documentation**: [https://docs.truefoundry.com/](https://docs.truefoundry.com/)
